The present invention relates to an image processing apparatus for a vehicle. More particularly, it relates to the image processing apparatus which is used, for example, as a steering controller of an unmanned automated vehicle or the like, for controlling the vehicle so that the vehicle detects lane markers (lane-indicating lines) on a road so as to track the lane markers.
In recent years, there prevails an image processing apparatus for a vehicle, which is carried on the vehicle, the apparatus being adapted to be capable of photographing a foreground in the vehicle-moving direction so as to obtain an image which corresponds to the foreground, processing the image, and then obtaining various information which is required to drive the vehicle or to control a steering device of the vehicle. More concretely, for example, there has been generally known an image processing apparatus, which is adapted to be capable of photographing the road in front of the vehicle to obtain an original image by using a digital image pick-up device, extracting an image, which corresponds to lane markers, from the original image, detecting position or attitude of the vehicle in the traffic lane based on the image of the lane markers, and then deciding target advance direction of the vehicle based on the detected result (for example, see Japanese Laid-open Patent Publication No. 3-137798).
Thus, in the conventional image processing apparatus which is disclosed in Japanese Laid-open Patent Publication No. 3-137798, for example, the image is processed so as to detect the position or attitude of the vehicle in the traffic lane, on the assumption that the helix angle (i.e. pitch angle) of the image pick-up device or vehicle is constant and further the height of the image pick-up device or vehicle against the road surface is constant (Hereinafter, this height is referred to xe2x80x9cimage pick-up heightxe2x80x9d.). Therefore, if the helix angle (i.e. pitch angle) or image pick-up height changes or fluctuates when the vehicle is moving, there may occur such a problem that error is accompanied with the detected position or attitude of the vehicle in the traffic lane. Thus, in order to solve the problem, there has been proposed an image processing apparatus, which is adapted to be capable of approximately expressing white lines in the vicinity of the vehicle in the original image photographed by the image pick-up device as a straight line so as to obtain a vanishing point (FOE), measuring the change of the helix angle (i.e. pitch angle) or image pick-up height based on the vanishing point, and then detecting the position or attitude of the vehicle in the traffic lane based on the change measured above (for example, see Japanese Laid-open Patent Publication No. 5-151340).
However, in the conventional image processing apparatus which measures the change of the helix angle (i.e. pitch angle) or image pick-up height based on the vanishing point, there exists such a problem that although the change of the helix angle (i.e. pitch angle) or image pick-up height of the vehicle can be exactly measured when each of the lane markers is approximately expressed as a straight line, the change of the helix angle (i.e. pitch angle) or image pick-up height cannot be exactly measured so that the position or attitude of the vehicle in the traffic lane cannot be exactly detected when each of the lane markers is expressed as a curved line, for example, due to winding of the road, because the vanishing point cannot be exactly detected.
The present invention has been developed to solve the conventional problems described above, and has an object of providing an image processing apparatus for a vehicle, which can exactly measure the change of the helix angle (i.e. pitch angle) of the vehicle or movement of the vehicle in the upper-lower direction (change of the image pick-up height) so as to be able to exactly detect the position or attitude of the vehicle in the traffic lane, even if each of the lane markers is expressed as a curved line.
According to the present invention which is achieved to solve the above-mentioned problems, there is provided an image processing apparatus for a vehicle, which is carried on the vehicle, the apparatus including, (a) an image pick-up device for photographing a foreground (a road and a scene around the road) in a vehicle-moving direction to generate an original image data which corresponds to the foreground, (b) an outline extracting device for extracting an outline image data from the original image data generated by the image pick-up device, the outline image data corresponding to outlines of two lane-indicating lines (lane markers) formed at both end portions in a width direction of a traffic lane on a road, on which the vehicle is moving, (c) a coordinate transforming device for performing a coordinate-transforming processing to the outline image data generated by the outline extracting device to generate a coordinate-transforming image data which represents a plan view of the outlines of the lane-indicating lines in a predetermined plane coordinate system, and (d) an attitude change detecting device for detecting a change of a helix angle (i.e. pitch angle) of the vehicle based on such a matter that the two lane-indicating lines are parallel to each other, by using the coordinate-transforming image data generated by the coordinate transforming device.
According to the image processing apparatus, from the image of the lane-indicating lines, a plane view of the lane-indicating lines (outlines) is formed by the coordinate transforming device. Because both lane-indicating lines, located at both end portions of the traffic lane respectively, are parallel to each other, the images of both lane-indicating lines are parallel to each other on the plan view, if the helix angle (i.e. pitch angle) of the vehicle is constant. However, if the helix angle (i.e. pitch angle) changes, the images of the lane-indicating lines are not parallel to each other on the plan view. For example, if the helix angle (i.e. pitch angle) or image pick-up height changes so that the position, where the optical axis of the image pick-up device crosses with the road surface, deviates forward from the normal position, the interval between the lane-indicating lines on the plan view becomes narrower as the position shifts forward. On the other hand, if the position, where the optical axis of the image pick-up device crosses with the road surface, deviates backward from the normal position, the interval between the lane-indicating lines on the plan view becomes wider as the position shifts forward. Therefore, the change of the helix angle (i.e. pitch angle) or image pick-up height can be measured based on the change of the interval of the lane-indicating lines on the plan view. Thus, in the image processing apparatus, if the lane-indicating lines (lane markers) are curved, the change of the helix angle (i.e. pitch angle) or image pick-up height can be detected by the attitude change detecting device, based on the change of the interval of the lane-indicating lines on the plan view. In consequence, the position or attitude of the vehicle in the traffic lane can be exactly detected.
In the image processing apparatus, it is preferable that (e) the plane coordinate system used by the coordinate transforming device is a plane coordinate in which a position of the image pick-up device is defined as an origin in a view point of plane, a width direction of the vehicle is defined as a first coordinate axis (X-axis), and a front-rear direction of the vehicle is defined as a second coordinate axis (Z-axis). Further, it is preferable that (f) the attitude change detecting device plots a plurality of points onto the plane coordinate, based on the coordinate-transforming image data, the position of each of the points in the first coordinate axis corresponding to the size in width of the traffic lane, and the position of the point in the second coordinate axis corresponding to the position in the longitudinal direction of the traffic lane. Moreover, it is preferable that the attitude change detecting device decides one straight line which matches or fits the points, and then detects the change of the helix angle (i.e. pitch angle) or the change of the amount of the movement of the vehicle in the upper-lower direction (the change of the image pick-up height), based on the inclination angle of the straight line against the second coordinate axis. In that case, the change of the helix angle (i.e. pitch angle) or image pick-up height can be easily measured by means of such a simple operation that nothing is required except calculation of the angle between the above-mentioned straight line and the second coordinate axis.
Further, it is more preferable that the image processing apparatus is provided with a target advance direction detecting device for detecting a target direction to advance the vehicle in consideration of at least one of the change of the helix angle (i.e. pitch angle) and the amount of movement of the vehicle in the upper-lower direction. In that case, the direction, along which the vehicle should be advanced or can be advanced, can be detected, and consequently the information can be effectively utilized for an automated steering device and so on.
Meanwhile, in the image processing apparatus, it is preferable that the attitude change detecting device judges outline points deviating from the above-mentioned straight line to be noises. In that case, because the noises are eliminated, the accuracy of measurement of the change of the helix angle (i.e. pitch angle) or image pick-up height may be improved.
The outline detecting device of the image processing apparatus, for example, may be concretely composed of (g) an edge extracting device for emphasizing edges of the traffic lane in the original image data generated by the image pick-up device, (h) a threshold setting device for setting a threshold to extract the edges of the traffic lane based on the result detected by the edge extracting device, (i) an outline point extracting device for extracting a pixel, whose intensity is a peek value, from pixels of the edges of the traffic lane, as an outline point data of the lane-indicating lines, based on both of the result detected by the edge extracting device and the threshold set by the threshold setting device, and (j) a lane-indicating line detecting device for performing an interpolation processing to the outline point data extracted by the outline point extracting device to generate the outline image data which corresponds to the outline of the lane-indicating lines.